QTM 385 - Experimental Methods

Lecture 06 - Texts for Discussion and More

Danilo Freire

Emory University

Hi, there! 👋
Tudo bem? 😄🇧🇷

Regression in R: Covered adding covariates with lm_lin(), centring, and sub-group analysis with interaction terms using lm_robust.
Hypothesis Testing: Covered different types of null hypotheses: sharp null and null hypothesis of no average treatment effect; confidence intervals, type I and type II errors
Statistical concept of sampling variability, how it relates to sampling distributions, and how it affects the interpretation of experimental estimates
Randomisation Inference: Introduced, with the ri2 package, which uses the randomizr package.
Demonstrated RI with two examples, including one about female council heads and the other about runoff elections.
Real Experiment: Analysed a paper using linear regression models as well as Randomisation Inference

Talk a bit about R and the packages we have seen so far
Discuss the next steps of your pre-analysis plan and the final project
Discuss three papers
- Kalla, J. & D. Broockman. 2016. Campaign Contributions Facilitate Access to Congressional Officials: A Randomized Field Experiment. American Journal of Political Science, 60(3): 545–558
- Chattopadhyay, R., & Duflo, E. (2004). Women as policy makers: Evidence from a randomized policy experiment in India. Econometrica, 72(5), 1409-1443.
- Shrout, P. E., & Rodgers, J. L. (2018). Psychology, science, and knowledge construction: Broadening perspectives from the replication crisis. Annual Review of Psychology, 69(1), 487-510.
But first…, let’s talk about your group work

Now that you have your groups, I’d like you to start working on your pre-analysis plan
I’ll give you some time to discuss your ideas and start writing your plan
My plan is that you should slowly think about your final project, one step at a time
What do you think about having two weeks to write the following:
- Submit at most 2 paragraphs summarising an experiment that you want to develop in this course. At minimum, your summary should include a research question, why the question is important, and a rough sketch of how you plan to answer the question.
In three weeks:
- Write a title and abstract for a paper you imagine writing based on your proposed experiment. Assume that your findings align with your theoretical predictions. Remember to establish why the findings matter for your intended audience.
In four weeks:
- Outline your pre-analysis plan. Your outline should include sections on the research question, the experimental design, the data you will collect, and the analysis you will conduct.

In five weeks:
- Use Quarto and DeclareDesign to write your report and simulate your experiment
In six weeks:
- Revised outline, now including a new section titled “Potential Threats.” In this section, diagnose threats and briefly describe potential countermeasures. This new section should discuss false positives, statistical power, demand effects, noncompliance, spillover, and attrition.
In seven weeks:
- Revised outline, now included a new section on “Heterogeneous Treatment Effects.” In this section, discuss how you might explore heterogeneity in treatment effects. This section should include a discussion of how you might use covariates to explore heterogeneity.
In eight weeks:
- I will give you feedback on your outline and provide you with the simulated data for your experiment.
- You will have two weeks to write your final report.
Last week of class:
- You will present your findings to the class.